
ISSN: 2249-0558 



MEMS-BASED DIGITAL PEN WITH A TRAJECTORY 
RECOGNITION ALGORITHM FOR HANDWRITTEN 
DIGIT AND ALPHABET RECOGNITION 



S. Iswarya, PG Student* 

Mr. T. Rajesh, Assistant professor 



Abstract — This paper presents an accelerometer-based digital pen for handwritten digit and 

gesture trajectory recognition applications. The digital pen consists of a triaxial accelerometer, a 
microcontroller (with A/D converter), and a ZigBee wireless transmission module for sensing 
and collecting accelerations of handwriting and gesture trajectories. The triaxial accelerometer 
measures the acceleration signals generated by a user's hand motions. The microcontroller 
collects the analog acceleration signals and converts the signals to digital ones via the A/D 
converter. Using this project we can do human computer interaction. Users can use the pen to 
write digits or make hand gestures, and the accelerations of hand motions measured by the 
accelerometer are wirelessly transmitted to a computer for online trajectory recognition. So, by 
changing the position of mems (micro electro mechanical systems) we can able to show the 
alphabetical characters in the PC. The acceleration signals measured from the triaxial 
accelerometer are transmitted to a computer via the wireless module. 
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I. INTRODUCTION 

The increase in human-machine interactions in our daily lives has made user interface 
technology progressively more important. Physical gestures as intuitive expressions will greatly 
ease the interaction process and enable humans to more naturally command computers or 
machines. Recently, an attractive alternative, a portable device embedded with inertial sensors, 
has been proposed to sense the activities of human and to capture his/her motion trajectory 
information from accelerations for recognizing gestures or handwriting. Use of inertial sensors 
has an advantage that they can be operated without any external reference and limitation in 
working conditions. However, motion trajectory recognition is relatively complicated because 
different users have different speeds and styles to generate various motion trajectories. 



In this work, a miniature MEMS accelerometer based recognition system which can recognize 
seven hand gestures in 3-D space is built. The system has potential uses such as a remote 
controller for visual and audio equipment, or as a control mechanism to Command machines and 
intelligent systems in offices and factories. There are mainly two existing types of gesture 
recognition methods, i.e., vision-based and accelerometer and/or gyroscope based. Due to the 
limitations such as unexpected ambient optical noise, slower dynamic response, and relatively 
large data collections/ processing of vision -based method, our recognition system is implemented 
based on an inertial measurement unit based on MEMS acceleration sensors. Since heavy 
computation burden will be brought if gyroscopes are used for inertial measurement, current 
system is based on MEMS accelerometers only and gyroscopes are not implemented for motion 
sensing. 

In this paper, developed a pen-type portable device and a trajectory recognition 
algorithm. The pen-type portable device consists of a triaxial accelerometer, a microcontroller, 
and an RF wireless transmission module. The acceleration signals measured from the triaxial 
accelerometer are transmitted to a computer via the wireless module. Users can utilize this digital 
pen to write digits and make hand gestures at normal speed. The measured acceleration signals of 
these motions can be recognized by the trajectory recognition algorithm. The recognition 
procedure is composed of acceleration acquisition, signal preprocessing, feature generation, 
feature selection, and feature extraction. The acceleration signals of hand motions are measured 
by the pen-type portable device. The signal preprocessing procedure consists of calibration, a 
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moving average filter, a high-pass filter, and normalization. First, the accelerations are calibrated 
to remove drift errors and offsets from the raw signals. These two filters are applied to remove 
high frequency noise and gravitational acceleration from the raw data, respectively. The features 
of the preprocessed acceleration signals of each axis include mean, root mean square (rms), 
VAR, standard deviation (STD), and energy. Before classifying the hand motion trajectories, that 
perform the procedures of feature selection and extraction methods. Therefore, the criterion of 
kernel-based class separability (KBCS) with best individual N (BIN) is to select significant 
features from the original features and that of linear discriminate analysis (LDA) is to reduce the 
dimension of the feature space with a better recognition performance. The objective of the 
feature selection and feature extraction methods is not only to ease the burden of computational 
load but also to increase the accuracy of classification. The reduced features are used as the 
inputs of classifiers. In this paper, we adopted a probabilistic neural network (PNN). The 
rest of this paper is organized as follows. In Sections II, the survey related work on digital pen 
trajectory recognition and similar HCI applications and introduce the hardware components of 
the digital pen in detail, respectively. The proposed trajectory recognition algorithm consisting of 
acceleration acquisition, signal preprocessing, feature generation, feature selection, feature 
extractions are presented. 

II. RELATED WORKS 

Existing gesture recognition approaches include template- matching [1], dictionary lookup [2], 
statistical matching [3], linguistic matching [4], and neural network [5]. For sequential data such 
as measurement of time series and acoustic features at successive time frames used for speech 
recognition, HMM (Hidden Markov Model) is one of the most important models [6] .It is 
effective for recognizing patterns with spatial and temporal variation [7]. In this paper, we 
present three different gesture recognition models, which are: 1) sign sequence and Hopfield 
based gesture recognition model; 2) velocity increment based gesture recognition model; and 3) 
sign sequence and template matching based gesture recognition model. In these three models, in 
order to find a simple and efficient solution to the hand gesture recognition problem based on 
MEMS accelerometers, the acceleration patterns are not mapped into velocity, displacement or 
transformed into frequency domain, but are directly segmented and recognized in time domain. 
By extracting a simple feature based on sign sequence of acceleration, the recognition system 
achieves high accuracy and efficiency without the employment of HMM. 
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Some studies have focused on the development of digital pens for trajectory recognition and HCI 
applications. For instance, an alternative method of conventional tablet -based handwriting 
recognition has been proposed by Milner [8]. In his system, two dual-axis accelerometers are 
mounted on the side of a pen to generate time-varying x- and y-axis acceleration for handwriting 
motion. The author employed an HMM with a band pass filtering and a down-sampling 
procedure for classification of seven handwritten words. The best recognition rate is 96.2% when 
the number of states of the HMM is equal to 60. Oh. [9] Presented a wand like input device 
embedding a triaxial accelerometer and a triaxial gyroscope for online 3-D character gesture 
recognition. Fisher discriminate analysis was adopted, and different combinations of sensor 
signals were used to test the recognition performance of their device. When all six axes raw 
signals were used as inputs of the recognition system, the recognition rate was 93.23%. In 
addition, they proposed an ensemble recognizer consisting of three sub recognizers with the 
following signals as inputs: acceleration, angular velocity, and estimated handwriting trajectory. 
The recognition rate of the recognizer was 95.04%. Similarly, a gesture recognition system 
consisting of a gesture input device, a trajectory estimation algorithm, and a recognition 
algorithm in 3-D space was proposed by Cho. [10]. The trajectory estimation algorithm based on 
an inertial navigation system was developed to reconstruct the trajectories of numerical digits 
and three hand gestures, and then, a Bayesian network was trained to recognize the reconstructed 
trajectories. The average recognition rate was 99.2%. Zhou. [11] Proposed a uIMU for 2-D 
handwriting applications. They extracted the discrete cosine transform features from x- and y- 
axis acceleration signals and one angular velocity and used an unsupervised self- organizing map 
to classify 26 English alphabets and ten numerical digits. The recognition rate of 26 English 
alphabets and ten numerical digits achieved 64.38% and 80.8%, respectively. 

III. SYSTEM DESIGN 
Digital pen consists of a triaxial accelerometer (LIS3L02AQ3), a microcontroller 
(PIC16F87A with a 12-b A/D converter), and a wireless transceiver. The triaxial accelerometer 
measures the acceleration signals generated by a user's hand motions. The microcontroller 
collects the analog acceleration signals and converts the signals to digital ones via the 
A/D converter. The wireless transceiver transmits the acceleration signals wirelessly to a 
personal computer (PC). The dimension of the pen-type circuit board is 14 cm x 2 cm x 1.5 cm. 
The LIS3L02AQ3 is a low-cost capacitive micro machined accelerometer with a temperature 
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B. 



compensation function. The accelerometer's sensitivity is set from -2 g to +2 g. The PIC16F87A 
integrates a high-performance 12-b A/D converter on a signal chip. The output signals of the 
accelerometer are sampled the 12-b A/D converter. Then, all the data sensed by the 
accelerometer are transmitted wirelessly to a PC by an RF transceiver. 

Sensor Description: The sensing system utilized in our experiments for hand motion data 
collection and is essentially a MEMS 3-axes acceleration sensing chip integrated with data 
management and ZIGBEE wireless data chips. The algorithms described in this paper were 
implemented and run on a PC. 

System work flow: When the sensing system is switched on, the accelerations in three 
perpendicular directions are detected by the MEMS sensors and transmitted to a PC via ZIGBEE 
protocol. 

In this project consist of mainly two sections, one is Pen section & second is PC section as 
shown in fig.l and fig. 2. PIC Microcontroller is control all the operation in this system.The 
triaxial accelerometer measures the acceleration signals generated by a user's hand motions. The 
microcontroller collects the analog acceleration signals and converts the signals to digital ones 
via the A/D converter. The wireless transceiver transmits the acceleration signals wirelessly to a 
personal computer (PC). The acceleration signals measured from the triaxial accelerometer are 
transmitted to a computer via the wireless module. 




Indicator 



PIC 

Micro 
controller 




ZigBee 
Transmitter 



Fig.l. PEN Section 



ZigBee 
receiver 



RS232 
Comm. 




Fig.2. PC Section 
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IV. TRAJECTORY RECOGNITION ALGORITHM 



The proposed trajectory recognition algorithm consisting of acceleration acquisition, 
signal preprocessing, feature generation, feature selection, and feature extraction. In this paper, 
the motions for recognition include numerals and hand gestures. The acceleration signals of the 
hand motions are measured by a triaxial accelerometer and then preprocessed by filtering and 
normalization. Consequently, the features are extracted from the preprocessed data to represent 
the characteristics of different motion signals, and the feature selection process based on KBCS. 
To reduce the computational load and increase the recognition accuracy of the classifier, uses 
LDA to reduce the dimension of the selected features. The reduced feature vectors are fed into a 
PNN classifier to recognize the motion to which the feature vector belongs. We now introduce 
the detailed procedure of the proposed trajectory recognition algorithm as follows. 

A. Signal Preprocessing: The raw acceleration signals of hand motions are generated by the 
accelerometer and collected by the microcontroller. Due to human nature, their hand always 
trembles slightly while moving, which causes certain amount of noise. The signal preprocessing 
consists of calibration, a moving average filter, a high-pass filter, and normalization. First, the 
accelerations are calibrated to remove drift errors and offsets from the raw signals. The second 
step of the signal preprocessing is to use a moving average filter to reduce the high-frequency 
noise of the calibrated accelerations, and the filter is expressed as 

N-l 

I If f PflffiH x[t+i ¥\ i*3 

i=l 

Where x[t] is the input signal, y[t] is the output signal, and N is the number of points in the 
average filter. 

B. Feature Generation: The characteristics of different hand movement signals can be obtained 
by extracting features from the preprocessed x-, y-, and z-axis signals, and can extract some 
features from the triaxial acceleration signals, including mean, STD, VAR, rms, and energy. 
They are explicated as follows. 

1) Mean: The mean value of the acceleration signals of each hand motion is the dc component of 
the signal 
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Mean 

where W is the length of each hand motion. 
2) STD: STD is the square root of VAR 



|W] 

= |W|2 Xi 



=i 



STD - 



I lw| 



3) VAR: 



VAR 



|W| 

= TT— rVCxi -m)2 
w — 1 Z_i 



1=1 



where xi is the acceleration instance and m is the mean value of xi. 
4) rms: 

ry |W| 

= JiwiZ Xl2 



rms 



i=l 



where xi is the acceleration instance and m is the mean value of xi 

5) Energy: Energy is calculated as the sum of the magnitudes of squared discrete fast Fourier 
transform (FFT) components of the signal in a window. The equation is defined as 

Energy = iwiZ |Fi|2 



i=l 



Where Fi is the ith FFT component of the window and IFil is the magnitude of Fi. When the 
procedure of feature generation is done, some features are then generated. Because the amount of 
the extracted features is large, we adopt KBCS to select most useful features and then use LDA 
to reduce the dimensions of features. 

C. Feature Selection: Feature selection comprises a selection criterion and a search strategy. 
The adopted selection criterion is the KBCS. The KBCS can be computed as follows: Let (x, y) 
G (Rd x Y ) represents a sample, where Rd denotes a d-dimensional feature space, Y symbolizes 
the set of class labels, and the size of Y is the number of class c. This method projects the 
samples onto a kernel space, and m0 i is defined as the mean vector for the ith class in the kernel 
space, ni denotes the number of samples in the ith class, m0 denotes the mean vector for all 
classes in the kernel space, S0 B denotes the between-class scatter matrix in the kernel space, 
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and S0 W denotes the within-class scatter matrix in the kernel space. Let 0( • ) be a possible 
nonlinear mapping from the feature space Rd to a kernel space k and tr(A) represents the trace of 
a square matrix A. The class separability in the kernel space can be measured as 

tr(sg) 



J = 



tr(S« 



To maintain the numerical stability in the maximization of J0, the denominator tr(S0 W) has to 
be prevented from approaching zero. 

D. Feature Extraction: For pattern recognition problems, LDA is an effective feature extraction 

which uses a linear transformation to transform the original feature sets into a lower dimensional 

feature space. The fundamental concept of LDA is to maximize the following Fisher criterion to 

search for the most efficient projection matrix W 

, W T S B w 
W'S w w 

in order to achieve maximal discrimination in the new feature space, transformation matrix W is 
utilized to maximize the ratio of the between-class distance to the within class distance. After 
feature extraction, these reduced features 



V. CONCLUSION 

Designed a MEMS -based digital pen for handwritten digit and alphabet recognition 
applications. The digital pen consists of a triaxial accelerometer, a microcontroller (with A/D 
converter), and an ZigBee wireless transmission module. Users can use the pen to write digits or 
make hand gestures, and the accelerations of hand motions measured by the accelerometer are 
wirelessly transmitted to a computer for online trajectory recognition. So, by changing the 
position of mems (micro electro mechanical systems) we can able to show the alphabetical 
characters in the PC. The proposed method is to be more efficient than the existing method. 
Moreover, using ZigBee technology, we can reduce the power and cost of the system. 
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